Combining multi-party speech and text exchanges over the internet

Bilateral or group text chatting over the Internet has become a favoured pastime for many people across the world. Yet it would seem that, in general, text chat is a severely impoverished mode of on-line communication compared to, e.g., fully situated human-human spoken conversation, video conferencing, or even speaking over the telephone. This paper explores what happens when on-line multi-speaker conversation over the Internet is added to text chat, creating what may become a widespread mode of communication in the near future. The system used is called the Magic Lounge. Magic Lounge offers a multimodal combination of text chat and spoken conversation for meetings and other encounters among ubiquitous users who may join the communication from workstations, PDAs and WAP phones. In addition, the system has a series of meeting history tools which provide various forms of structure to the spoken and text chat records of the meeting as it unfolds and after the meeting. The paper presents rather clear-cut results on the respective communicative roles of speech and text chat from a series of user tests with the system in which different groups of users performed scenarios designed to explore the combined use of text chat and speech. The results reported may generalise to a wide range of applications which combine text and spoken information representation.